Development and internal validation of machine learning-based models and external validation of existing risk scores for outcome prediction in patients with ischaemic stroke.

Aims: We developed new machine learning (ML) models and externally validated existing statistical models [ischaemic stroke predictive risk score (iScore) and totalled health risks in vascular events (THRIVE) scores] for predicting the composite of recurrent stroke or all-cause mortality at 90 days and at 3 years after hospitalization for first acute ischaemic stroke (AIS). Methods and results: In adults hospitalized with AIS from January 2005 to November 2016, with follow-up until November 2019, we developed three ML models [random forest (RF), support vector machine (SVM), and extreme gradient boosting (XGBOOST)] and externally validated the iScore and THRIVE scores for predicting the composite outcomes after AIS hospitalization, using data from 721 patients and 90 potential predictor variables. At 90 days and 3 years, 11 and 34% of patients, respectively, reached the composite outcome. For the 90-day prediction, the area under the receiver operating characteristic curve (AUC) was 0.779 for RF, 0.771 for SVM, 0.772 for XGBOOST, 0.720 for iScore, and 0.664 for THRIVE. For 3-year prediction, the AUC was 0.743 for RF, 0.777 for SVM, 0.773 for XGBOOST, 0.710 for iScore, and 0.675 for THRIVE. Conclusion: The study provided three ML-based predictive models that achieved good discrimination and clinical usefulness in outcome prediction after AIS and broadened the application of the iScore and THRIVE scoring system for long-term outcome prediction. Our findings warrant comparative analyses of ML and existing statistical method-based risk prediction tools for outcome prediction after AIS in new data sets.

Trimethoprim-sulfamethoxazole significantly reduces the risk of nocardiosis in solid organ transplant recipients: systematic review and individual patient data meta-analysis.

BACKGROUND: Whether trimethoprim-sulfamethoxazole (TMP-SMX) prophylaxis prevents nocardiosis in solid organ transplant (SOT) recipients is controversial. OBJECTIVES: To assess the effect of TMP-SMX in the prevention of nocardiosis after SOT, its dose-response relationship, its effect on preventing disseminated nocardiosis, and the risk of TMP-SMX resistance in case of breakthrough infection. METHODS: A systematic review and individual patient data meta-analysis. DATA SOURCES: MEDLINE, Embase, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, Web of Science Core Collection, and Scopus up to 19 September 2023. STUDY ELIGIBILITY CRITERIA: (a) Risk of nocardiosis between SOT recipients with and without TMP-SMX prophylaxis, or (b) sufficient details to determine the rate of TMP-SMX resistance in breakthrough nocardiosis. PARTICIPANTS: SOT recipients. INTERVENTION: TMP-SMX prophylaxis versus no prophylaxis. ASSESSMENT OF RISK OF BIAS: Risk Of Bias In Non-randomized Studies-of Exposure (ROBINS-E) for comparative studies; dedicated tool for non-comparative studies. METHODS OF DATA SYNTHESIS: For our primary outcome (i.e. to determine the effect of TMP-SMX on the risk of nocardiosis), a one-step mixed-effects regression model was used to estimate the association between the outcome and the exposure. Univariate and multivariable unconditional regression models were used to adjust for the potential confounding effects. Certainty of evidence was assessed using Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. RESULTS: Individual data from three case-control studies were obtained (260 SOT recipients with nocardiosis and 519 uninfected controls). TMP-SMX prophylaxis was independently associated with a significantly decreased risk of nocardiosis (adjusted OR = 0.3, 95% CI 0.18-0.52, moderate certainty of evidence). Variables independently associated with an increased risk of nocardiosis were older age, current use of corticosteroids, high calcineurin inhibitor concentration, recent acute rejection, lower lymphocyte count, and heart transplant. Breakthrough infections (66/260, 25%) were generally susceptible to TMP-SMX (pooled proportion 98%, 95% CI 92-100). CONCLUSIONS: In SOT recipients, TMP-SMX prophylaxis likely reduces the risk of nocardiosis. Resistance appears uncommon in case of breakthrough infection.

Impact of Size Thresholds on the Diagnosis of Incidental Adrenal Lesions: A Systematic Review and Meta-Analysis.

BACKGROUND: Preferred size-threshold recommendations for management of incidental adrenal lesions remain controversial. PURPOSE: This meta-analysis aimed to compare the diagnostic accuracy of different size thresholds for detecting malignancy in patients with incidental adrenal lesions on imaging. MATERIALS AND METHODS: A systematic review of MEDLINE, Embase, Scopus, the Cochrane Library, and the gray literature, covering the period from inception to September 2021, was performed. Studies with >10 patients evaluating the diagnostic accuracy of imaging size thresholds for detecting malignancy in patients with incidental adrenal lesions and no prior history of cancer were included. Study, clinical, imaging, and accuracy data for eligible studies were independently acquired by two reviewers. Primary meta-analysis was performed using a bivariate mixed-effects regression model. Risk of bias was evaluated using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. RESULTS: From 2,690 citations, 40 studies (9,794 patients with mean age ranging from 41 to 66 years) were included. Most (36 of 40) were retrospective single-center studies. CT with or without MRI served as the index test(s). Sensitivity and specificity values, respectively, by size threshold used in the included studies were as follows: 85% (95% confidence interval [CI] 74%-91%) and 39% (95% CI 23%-57%) for 3-cm thresholds; 85% (95% CI 78%-90%) and 75% (95% CI 62%-85%) for 4-cm thresholds; 70% (95% CI 56%-81%) and 74% (95% CI 59%-85%) for 5-cm thresholds; and 75% (95% CI 67%-82%) and 77% (95% CI 62%-87%) for 6-cm thresholds. No cause for variability in sensitivity or specificity was identified on subgroup analysis of the 4-cm threshold. Nearly half of the studies (19 of 40) had at least one QUADAS-2 domain with a high risk of bias. CONCLUSIONS: A 4-cm size threshold demonstrates the highest combined sensitivity and specificity, with a preserved specificity compared with higher size thresholds, but with a trend toward improved sensitivity. Future research reevaluating 4-5 cm size thresholds while excluding characteristically benign lesions by imaging may help redefine a size threshold that has improved specificity but preserved sensitivity, compared with the existing 4-cm threshold.

Diagnostic accuracy of CT and MR features for detecting atypical lipomatous tumors and malignant liposarcomas: a systematic review and meta-analysis.

OBJECTIVES: This systematic review and meta-analysis evaluated the diagnostic accuracy of CT and MRI for differentiating atypical lipomatous tumors and malignant liposarcomas from benign lipomatous lesions. METHODS: MEDLINE, EMBASE, Scopus, the Cochrane Library, and the gray literature from inception to January 2022 were systematically evaluated. Original studies with > 5 patients evaluating the accuracy of CT and/or MRI for detecting liposarcomas with a histopathological reference standard were included. Meta-analysis was performed using a bivariate mixed-effects regression model. Risk of bias was evaluated using Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). This study is registered on PROSPERO, number CRD42022306479. RESULTS: Twenty-six studies with a total of 2613 patients were included. Mean/median reported patient ages ranged between 50 and 63 years. The summary sensitivity and specificity of radiologist gestalt for detecting liposarcomas was 85% (79-90% 95% CI) and 63% (52-72%), respectively. Deep depth to fascia, thickened septations, enhancing components, and lesion size (≥ 10 cm) all demonstrated sensitivities ≥ 85%. Other imaging characteristics including heterogenous/amorphous signal intensity, irregular tumor margin, and nodules present demonstrated lower sensitivities ranging from 43 to 65%. Inter-reader reliability for radiologist gestalt within studies ranged from fair to substantial (k = 0.23-0.7). Risk of bias was predominantly mixed for patient selection, low for index test and reference standard, and unclear for flow and timing. CONCLUSION: Higher sensitivities for detecting liposarcomas were achieved with radiologist gestalt, deep depth to fascia, thickened septations, enhancing components, and large size. Combined clinical and imaging scoring and/or radiomics both show promise for optimal performance, though require further analysis with prospective study designs. CLINICAL RELEVANCE: This pooled analysis evaluates the accuracy of CT and MRI for detecting atypical lipomatous tumors and malignant liposarcomas. Radiologist gestalt, deep depth to fascia, thickened septations, enhancing components, and large size demonstrate the highest overall sensitivities. KEY POINTS: • The summary sensitivity and specificity of radiologist gestalt for detecting liposarcomas was 85% (79-90% 95% CI) and 63% (52-72%), respectively. • Radiologist gestalt, deep depth to fascia, thickened septations, enhancing components, and large tumor size (≥ 10 cm) showed the highest sensitivities for detecting atypical lipomatous tumors/well-differentiated liposarcomas and malignant liposarcomas. • A combined clinical and imaging scoring system and/or radiomics is likely to provide the best overall diagnostic accuracy, although currently proposed scoring systems and radiomic feature analysis require further study with prospective study designs.

Lack of patient-centered evaluation of outcomes in intermittent claudication literature.

BACKGROUND: Peripheral arterial disease, characterized as arterial atherosclerotic disease, can lead to insufficient flow in the lower extremities and ischemia, with the most common clinical manifestation being intermittent claudication (IC). In 2022, the Society for Vascular Surgery (SVS) developed appropriate use criteria for the management of IC that used this systematic review as a source of evidence. The objective of this study is to synthesize the findings of the systematic review and identify evidence gaps. METHODS: A comprehensive search of literature databases including MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, and Scopus was conducted from January 1, 2000, to November 30, 2022. Noncomparative and comparative observational studies and randomized controlled trials were included. Included studies evaluated exercise therapy, endovascular or open revascularization for the treatment of IC. Outcomes of interest (freedom from major adverse limb event, health-related quality of life, and walking distance) were compared in various subgroups (age, sex, diabetes, smoking status, anatomical location of disease, and optimal medical therapy). RESULTS: Twenty-six studies reported the outcomes of interest for the evidence map. The general conclusions of the studies that reported freedom from major adverse limb events were that reintervention rates for endovascular therapy at ≥2 years were >20%, major amputation rates were often not reported, and, after endovascular therapy, the 1-month mortality was low (<2%). Quality of life and walking distance data were sparse, limited to only endovascular intervention, and insufficient to make any strong conclusions. CONCLUSIONS: IC in patients with peripheral arterial disease poses a significant socioeconomic and health care burden. Major, consequential gaps exist in the IC literature with respect to the assessment of patient reported outcome measures, standardized measures of walking distance and the comparative effectiveness of initial exercise therapy vs invasive intervention. The evidence gaps identified by the Society for Vascular Surgery appropriate use criteria on IC systematic review serve as a guide for future research efforts to optimize care for this patient population.

An artificial intelligence approach for predicting death or organ failure after hospitalization for COVID-19: development of a novel risk prediction tool and comparisons with ISARIC-4C, CURB-65, qSOFA, and MEWS scoring systems.

BACKGROUND: We applied machine learning (ML) algorithms to generate a risk prediction tool [Collaboration for Risk Evaluation in COVID-19 (CORE-COVID-19)] for predicting the composite of 30-day endotracheal intubation, intravenous administration of vasopressors, or death after COVID-19 hospitalization and compared it with the existing risk scores. METHODS: This is a retrospective study of adults hospitalized with COVID-19 from March 2020 to February 2021. Patients, each with 92 variables, and one composite outcome underwent feature selection process to identify the most predictive variables. Selected variables were modeled to build four ML algorithms (artificial neural network, support vector machine, gradient boosting machine, and Logistic regression) and an ensemble model to generate a CORE-COVID-19 model to predict the composite outcome and compared with existing risk prediction scores. The net benefit for clinical use of each model was assessed by decision curve analysis. RESULTS: Of 1796 patients, 278 (15%) patients reached primary outcome. Six most predictive features were identified. Four ML algorithms achieved comparable discrimination (P > 0.827) with c-statistics ranged 0.849-0.856, calibration slopes 0.911-1.173, and Hosmer-Lemeshow P > 0.141 in validation dataset. These 6-variable fitted CORE-COVID-19 model revealed a c-statistic of 0.880, which was significantly (P < 0.04) higher than ISARIC-4C (0.751), CURB-65 (0.735), qSOFA (0.676), and MEWS (0.674) for outcome prediction. The net benefit of the CORE-COVID-19 model was greater than that of the existing risk scores. CONCLUSION: The CORE-COVID-19 model accurately assigned 88% of patients who potentially progressed to 30-day composite events and revealed improved performance over existing risk scores, indicating its potential utility in clinical practice.

Diagnostic error among vulnerable populations presenting to the emergency department with cardiovascular and cerebrovascular or neurological symptoms: a systematic review.

BACKGROUND: Diagnostic error (DE) is a common problem in clinical practice, particularly in the emergency department (ED) setting. Among ED patients presenting with cardiovascular or cerebrovascular/neurological symptoms, a delay in diagnosis or failure to hospitalise may be most impactful in terms of adverse outcomes. Minorities and other vulnerable populations may be at higher risk of DE. We aimed to systematically review studies reporting the frequency and causes of DE in under-resourced patients presenting to the ED with cardiovascular or cerebrovascular/neurological symptoms. METHODS: We searched EBM Reviews, Embase, Medline, Scopus and Web of Science from 2000 through 14 August 2022. Data were abstracted by two independent reviewers using a standardised form. The risk of bias (ROB) was assessed using the Newcastle-Ottawa Scale, and the certainty of evidence was evaluated using the Grading of Recommendations Assessment, Development, and Evaluation approach. RESULTS: Of the 7342 studies screened, we included 20 studies evaluating 7436,737 patients. Most studies were conducted in the USA, and one study was multicountry. 11 studies evaluated DE in patients with cerebrovascular/neurological symptoms, 8 studies with cardiovascular symptoms and 1 study examined both types of symptoms. 13 studies investigated missed diagnoses and 7 studies explored delayed diagnoses. There was significant clinical and methodological variability, including heterogeneity of DE definitions and predictor variable definitions as well as methods of DE assessment, study design and reporting.Among the studies evaluating cardiovascular symptoms, black race was significantly associated with higher odds of DE in 4/6 studies evaluating missed acute myocardial infarction (AMI)/acute coronary syndrome (ACS) diagnosis compared with white race (OR from 1.18 (1.12-1.24) to 4.5 (1.8-11.8)). The association between other analysed factors (ethnicity, insurance and limited English proficiency) and DE in this domain varied from study to study and was inconclusive.Among the studies evaluating DE in patients with cerebrovascular/neurological symptoms, no consistent association was found indicating higher or lower odds of DE. Although some studies showed significant differences, these were not consistently in the same direction.The overall ROB was low for most included studies; however, the certainty of evidence was very low, mostly due to serious inconsistency in definitions and measurement approaches across studies. CONCLUSIONS: This systematic review demonstrated consistent increased odds of missed AMI/ACS diagnosis among black patients presenting to the ED compared with white patients in most studies. No consistent associations between demographic groups and DE related to cerebrovascular/neurological diagnoses were identified. More standardised approaches to study design, measurement of DE and outcomes assessment are needed to understand this problem among vulnerable populations. TRIAL REGISTRATION NUMBER: The study protocol was registered in the International Prospective Register of Systematic Reviews PROSPERO 2020 CRD42020178885 and is available from: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020178885.

The use of MRI, PET/CT, and nuclear scintigraphy in the imaging of pyogenic native vertebral osteomyelitis: a systematic review and meta-analysis.

BACKGROUND CONTEXT: Native vertebral osteomyelitis (NVO) is a severe infection with an increasing incidence globally. Although there is no widely agreed upon reference standard for diagnosis of the disease, imaging plays a crucial role. Magnetic resonance imaging (MRI) is currently the imaging modality of choice. In recent years, advances in imaging have allowed for a larger role for alternative imaging techniques in the setting of NVO. PURPOSE: Our aim was to evaluate the diagnostic accuracy of MRI, PET/CT, and nuclear imaging, namely 67Gallium and 99mTechnetium scintigraphy, in the diagnosis of pyogenic NVO. STUDY DESIGN/SETTING: We conducted a systematic review of five medical databases and included all studies from 1970 to September 2021 that compared imaging techniques and provided sufficient data for diagnostic test accuracy meta-analysis. METHODS: Abstract screening, full text review, and data extraction were done by a pair of independent reviewers. Nonnative and nonpyogenic patients were excluded. A bivariate random effect model was used for meta-analysis. RESULTS: Twenty studies were included in the meta-analysis, encompassing a total of 1,123 imaging studies. The meta-analysis sensitivity and specificity of MRI were 90% and 72% respectively; those of PET/CT were 93% and 80%; those of 67Ga were 95% and 88%; those of 99mTc were 86% and 39%; and the sensitivity and specificity of combined Ga and Tc were 91% and 92% respectively in the setting of suspected NVO. CONCLUSIONS: 67Ga has the highest sensitivity for NVO, and its specificity is augmented when combined with 99mTc. MRI and PET/CT are both highly sensitive modalities, although the specificity of PET/CT is slightly better. MRI remains an appropriate initial test depending on the availability of other modalities.

Smoking Cessation Interventions in Indigenous North Americans: A Meta-Narrative Systematic Review.

INTRODUCTION: Indigenous North Americans have the highest cigarette smoking prevalence among all racial and ethnic groups in the United States. We seek to identify effective components of smoking cessation interventions in Indigenous people in the United States associated with favorable cessation outcomes. METHODS: A review of literature studying smoking cessation interventions in Indigenous North Americans (American Indians and Alaska Natives) from January 2010 through August 2021 was completed. The primary objective of this study was to identify components of interventions associated with positive smoking cessation outcomes in Indigenous people. The studies identified were synthesized in a meta-narrative approach. RESULTS: Ten studies out of 608 titles were included (6 randomized trials, 2 single-arm studies, 1 cohort study, and 1 prospective observational study). Five categories of smoking cessation interventions were identified; phone or web-based tools, culturally-tailored interventions, the inclusion of Indigenous study personnel, pharmaceutical cessation aids, and behavioral health interventions. Phone and web tools, cultural tailoring, and inclusion of Indigenous personnel conditions inconsistently influenced smoking cessation. Pharmaceutical aids were viewed favorably among participants. Individualized behavioral counseling sessions were effective at promoting smoking cessation, as was input from local communities in the planning and implementation phases of study. CONCLUSION: A successful smoking cessation intervention in Indigenous North Americans includes Tribal or community input in intervention design and implementation; should provide individualized counseling sessions for participants, and offer access to validated smoking cessation tools including pharmacotherapy. IMPLICATIONS: This study identifies a paucity of smoking interventions utilizing standard of care interventions in Indigenous North Americans. Standard of care interventions including individualized cessation counseling and pharmacotherapy were effective at promoting cessation. The use of novel culturally tailored cessation interventions was not more effective than existing evidence-based care with the exception of including Tribal and local community input in intervention implementation. Future smoking cessation interventions in Indigenous North Americans should prioritize the use of standard of care cessation interventions.

Association between basic life support and survival in sports-related sudden cardiac arrest: a meta-analysis.

AIMS: To evaluate the association of basic life support with survival after sports-related sudden cardiac arrest (SR-SCA). METHODS AND RESULTS: In this systematic review and meta-analysis, a search of several databases from each database inception to 31 July 2021 without language restrictions was conducted. Studies were considered eligible if they evaluated one of three scenarios in patients with SR-SCA: (i) bystander presence, (ii) bystander cardiopulmonary resuscitation (CPR), or (iii) bystander automated external defibrillator (AED) use and provided information on survival. Risk of bias was evaluated using Risk of Bias in Non-randomized Studies of Interventions. The primary outcome was survival at the longest follow up. The meta-analysis was conducted using the random-effects model. The Grading of Recommendations Assessment, Development, and Evaluations (GRADE) approach was used to rate certainty in the evidence. In total, 28 non-randomized studies were included. The meta-analysis showed significant benefit on survival in all three groups: bystander presence [odds ratio (OR) 2.55, 95% confidence interval (CI) 1.48-4.37; I2 = 25%; 9 studies-988 patients], bystander CPR (OR 3.84, 95% CI 2.36-6.25; I2 = 54%; 23 studies-2523 patients), and bystander AED use (OR 5.25, 95% CI 3.58-7.70; I2 = 16%; 19 studies-1227 patients). The GRADE certainty of evidence was judged to be moderate. CONCLUSION: In patients with SR-SCA, bystander presence, bystander CPR, and bystander AED use were significantly associated with survival. These results highlight the importance of witness intervention and encourage countries to develop their first aid training policy and AED installation in sport settings.

Patient and Physician Decisional Factors Regarding Hypercalcemia of Malignancy Treatment: A Novel Mixed-Methods Study.

BACKGROUND: Integrating shared decision making between patients and physicians and incorporating their values and preferences in the development of clinical practice guidelines (CPGs) is of critical importance to optimize CPG implementation and treatment adherence. This applies to many debilitating diseases, including hypercalcemia of malignancy (HCM). OBJECTIVE: Evaluate patient and physician values, preferences, and attitudes to better inform CPGs to treat HCM in adults. METHODS: We followed a mixed-methods approach. We conducted a systematic review using 5 databases to identify studies reporting on patient and physician values, costs and resources, feasibility, acceptability, and equity regarding HCM treatment. We also gathered data from different countries on the cost of multiple treatment modalities. We collected data on outcome prioritization from the CPG Working Group. Similarly, we collected data from patients with HCM regarding outcome prioritization and administered a questionnaire to evaluate their attitudes and perceptions toward treatment as well as treatment acceptability and feasibility. RESULTS: In the systematic review, we included 2 cross-sectional surveys conducted on the same population of physicians who agreed that treating HCM alleviates symptoms and improves quality of life; however, harms and benefits should be thoroughly considered when deciding on the duration of treatment. We also included 2 studies on cost showing that intravenous (IV) bisphosphonate is more cost-effective than a combination of IV bisphosphonate and calcitonin and administration of IV zoledronic acid at home is more cost-effective than other IV bisphosphonates. The cost of zoledronic acid, denosumab, and cinacalcet varied widely among countries and types (brand vs generic). Both the CPG Working Group and patients with HCM agreed that the most important outcomes when deciding on treatment were survival and resolution of HCM, but there was some variability in the ratings for other outcomes. CONCLUSION: Using mixed methods, CPG developers can obtain meaningful information regarding evidence to decision criteria. In the case of HCM CPGs, this approach has provided the required contextual information and supported the development of evidence-based recommendations.

Early switch to oral antibiotic therapy for the treatment of patients with bacterial native vertebral osteomyelitis: a quaternary center experience, systematic review, and meta-analysis.

Recent data suggest that oral therapy can be effective for bone infections. We aim to assess the efficacy of an early switch to oral therapy ( < 2  weeks) compared to a non-early switch in bacterial native vertebral osteomyelitis. We conducted a cohort study at Mayo Clinic, Rochester (MN), between 2019-2021 combined with a systematic review, which queried multiple databases. Data were analyzed using a random-effects model. The cohort study included 139 patients: two received an early switch. Of 3708 citations, 13 studies were included in the final analysis. Meta-analysis demonstrated no difference in treatment failure (odds ratio  =  1.073, 95 % confidence interval 0.370-3.116), but many studies presented high risk of bias. Current evidence is insufficient to conclude the proportion of patients with failure or relapse is different in the two groups. High-quality studies are warranted before early switch can be routinely recommended.

GRADE Guidance 34: update on rating imprecision using a minimally contextualized approach.

OBJECTIVES: The aim of this study is to provide updated guidance on when The Grading of Recommendations Assessment, Development and Evaluation (GRADE) users should consider rating down more than one level for imprecision using a minimally contextualized approach. STUDY DESIGN AND SETTING: Based on the first GRADE guidance addressing imprecision rating in 2011, a project group within the GRADE Working Group conducted iterative discussions and presentations at GRADE Working Group meetings to produce this guidance. RESULTS: GRADE suggests aligning imprecision criterion for systematic reviews and guidelines using the approach that relies on thresholds and confidence intervals (CI) of absolute effects as a primary criterion for imprecision rating (i.e., CI approach). Based on the CI approach, when a CI appreciably crosses the threshold(s) of interest, one should consider rating down two or three levels. When the CI does not cross the threshold(s) and the relative effect is large, one should implement the optimal information size (OIS) approach. If the sample size of the meta-analysis is far less than the OIS, one should consider rating down more than one level for imprecision. CONCLUSION: GRADE provides updated guidance for imprecision rating in a minimally contextualized approach, with a focus on the circumstances in which one should seriously consider rating down two or three levels for imprecision.

GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines.

BACKGROUND AND OBJECTIVE: This is the 24th in the ongoing series of articles describing the GRADE approach for assessing the certainty of a body of evidence in systematic reviews and health technology assessments and how to move from evidence to recommendations in guidelines. METHODS: Guideline developers and authors of systematic reviews and other evidence syntheses use randomized controlled studies (RCTs) and non-randomized studies of interventions (NRSI) as sources of evidence for questions about health interventions. RCTs with low risk of bias are the most trustworthy source of evidence for estimating relative effects of interventions because of protection against confounding and other biases. However, in several instances, NRSI can still provide valuable information as complementary, sequential, or replacement evidence for RCTs. RESULTS: In this article we offer guidance on the decision regarding when to search for and include either or both types of studies in systematic reviews to inform health recommendations. CONCLUSION: This work aims to help methodologists in review teams, technology assessors, guideline panelists, and anyone conducting evidence syntheses using GRADE.

Peripartum Antibiotics Exposure and the Risk of Autoimmune and Autism Disorders in the Offspring.

Background As the use of antibiotics during the peripartum period increases, the incidence of autoimmune disorders and autism spectrum disorders (ASDs) is also increasing. In this study, we aim to assess if antibiotic exposure during the peripartum period affects the incidence of autoimmune diseases and ASD in the offspring. Methods We identified children (< 18 years of age) born in Olmsted County from January 1, 2003 through December 31, 2012. Offspring with celiac disease (CD), inflammatory bowel disease (IBD), or ASD diagnoses were matched to two controls on birth date, index date, mother's age at delivery, and sex. Data from the mother's medical records were retrieved to determine peripartum antibiotics use. Results A total of 242 cases and 484 matched controls were included in this study. Median age at the last follow-up was 11.3 years (range: 0.5-14.9), 73% were males in both groups. Odds of CD diagnosis was not statistically different between vaginal delivery with antibiotics compared with vaginal delivery with no antibiotics (odds ratio [OR] = 0.76, 95% confidence interval [CI]: 0.32-1.85), similarly in IBD (OR = 2.41, 95% CI: 0.53-10.98) and ASD (OR = 1.00, 95% CI:0.55-1.79). Preeclampsia or eclampsia was associated with offspring CD (OR = 3.20, 95% CI: 1.05-9.78). Smoking history and diabetes mellitus were associated with offspring ASD (OR = 1.84, 95% CI: 1.22-2.77 and OR = 2.01, 95% CI: 1.03-3.91, respectively). Conclusion In this cohort, we found no statistically significant association between peripartum antibiotics exposure and the development of CD, IBD, or ASD.

Adjuvant Tyrosine Kinase Inhibitors in Renal Cell Carcinoma: A Concluded Living Systematic Review and Meta-Analysis.

PURPOSE: Multiple large clinical trials have investigated adjuvant tyrosine kinase inhibitors (TKIs) to reduce the risk of cancer recurrence and progression to metastasis in high-risk renal cell carcinoma. We sought to maintain living and interactive evidence on this topic, until a high level of certainty is reached for key clinical outcomes such that further updates become unnecessary and unlikely to change clinical practice. METHODS: We created a living interactive evidence synthesis platform to maintain a continuously updated meta-analysis on TKI monotherapy in adjuvant renal cell carcinoma. We implemented an automated search strategy with weekly updates to identify randomized phase 2 and 3 clinical trials. Study selection, appraisal, and data extraction were done in duplicate. Cumulative meta-analysis was performed using Analyzer Module in Living Interactive Evidence platform. For each outcome (overall survival [OS], disease-free survival [DFS], and all-cause and treatment-related adverse events), we assessed certainty of evidence using GRADE approach and conducted trial sequential analysis. RESULTS: This final update includes five randomized trials including recently updated data from PROTECT trial. Meta-analysis shows that adjuvant TKI monotherapy offers no benefit in OS (hazard ratio, 1.01; 95% CI, 0.91 to 1.12, high certainty) or DFS (hazard ratio, 0.92; 95% CI, 0.86 to 1.00, high certainty) and significantly increases adverse event risk. Lack of benefit was consistent across subgroups including highest-risk patients (test for subgroup differences: P = .32). Optimal information size criteria were met, and there was high certainty of evidence for lack of DFS and OS benefit for adjuvant TKIs. CONCLUSION: There is no guidance on when to stop maintaining a living review. In this example, we used trial sequential analysis and high certainty of evidence (future clinical trials unlikely to change current conclusions) as a benchmark to conclude a living review in view of convincing evidence.

A Living, Interactive Systematic Review and Network Meta-analysis of First-line Treatment of Metastatic Renal Cell Carcinoma.

CONTEXT: Identifying the most effective first-line treatment for metastatic renal cell carcinoma (mRCC) is challenging as rapidly evolving data quickly outdate the existing body of evidence, and current approaches to presenting the evidence in user-friendly formats are fraught with limitations. OBJECTIVE: To maintain living evidence for contemporary first-line treatment for previously untreated mRCC. EVIDENCE ACQUISITION: We have created a living, interactive systematic review (LISR) and network meta-analysis for first-line treatment of mRCC using data from randomized controlled trials comparing contemporary treatment options with single-agent tyrosine kinase inhibitors. We applied an advanced programming and artificial intelligence-assisted framework for evidence synthesis to create a living search strategy, facilitate screening and data extraction using a graphical user interface, automate the frequentist network meta-analysis, and display results in an interactive manner. EVIDENCE SYNTHESIS: As of October 22, 2020, the LISR includes data from 14 clinical trials. Baseline characteristics are summarized in an interactive table. The cabozantinib + nivolumab combination (CaboNivo) is ranked the highest for the overall response rate, progression-free survival, and overall survival, whereas ipilimumab + nivolumab (NivoIpi) is ranked the highest for achieving a complete response (CR). NivoIpi, and atezolizumab + bevacizumab (AteBev) were ranked highest (lowest toxicity) and CaboNivo ranked lowest for treatment-related adverse events (AEs). Network meta-analysis results are summarized as interactive tables and plots, GRADE summary-of-findings tables, and evidence maps. CONCLUSIONS: This innovative living and interactive review provides the best current evidence on the comparative effectiveness of multiple treatment options for patients with untreated mRCC. Trial-level comparisons suggest that CaboNivo is likely to cause more AEs but is ranked best for all efficacy outcomes, except NivoIpi offers the best chance of CR. Pembrolizumab + axitinib and NivoIpi are acceptable alternatives, except NivoIpi may not be preferred for patients with favorable risk. Although network meta-analysis provides rankings with statistical adjustments, there are inherent biases in cross-trial comparisons with sparse direct evidence that does not replace randomized comparisons. PATIENT SUMMARY: It is challenging to decide the best option among the several treatment combinations of immunotherapy and targeted treatments for newly diagnosed metastatic kidney cancer. We have created interactive evidence summaries of multiple treatment options that present the benefits and harms and evidence certainty for patient-important outcomes. This evidence is updated as soon as new studies are published.

GRADE guidelines 32: GRADE offers guidance on choosing targets of GRADE certainty of evidence ratings.

OBJECTIVE: To provide practical principles and examples to help GRADE users make optimal choices regarding their ratings of certainty of evidence using a minimally or partially contextualized approach. STUDY DESIGN AND SETTING: Based on the GRADE clarification of certainty of evidence in 2017, a project group within the GRADE Working Group conducted iterative discussions and presentations at GRADE Working Group meetings to refine this construct and produce practical guidance. RESULTS: Systematic review and health technology assessment authors need to clarify what it is in which they are rating their certainty of evidence (i.e., the target of their certainty rating). The decision depends on the degree of contextualization (partially or minimally contextualized), thresholds (null, small, moderate or large effect threshold), and where the point estimate lies in relation to the chosen threshold(s). When the 95% confidence interval crosses multiple possible thresholds (i.e., including both large benefit and large harm), it is not worthwhile for authors to determine the target of certainty rating. CONCLUSION: GRADE provides practical principles to help systematic review and health technology assessment authors specify the target of their certainty of evidence rating.

Diagnostic Accuracy of MRI for the Detection of Malignant Peripheral Nerve Sheath Tumors: A Systematic Review and Meta-Analysis.

OBJECTIVE. This systematic review and meta-analysis evaluates the diagnostic accuracy of MRI for differentiating malignant (MPNSTs) from benign peripheral nerve sheath tumors (BPNSTs). MATERIALS AND METHODS. A systematic review of MEDLINE, Embase, Scopus, the Cochrane Library, and the gray literature from inception to December 2019 was performed. Original articles that involved at least 10 patients and that evaluated the accuracy of MRI for detecting MPNSTs were included. Two reviewers independently extracted clinical and radiologic data from included articles to calculate sensitivity, specificity, PPV, NPV, and accuracy. A meta-analysis was performed using a bivariate mixed-effects regression model. Risk of bias was evaluated using QUADAS-2. RESULTS. Fifteen studies involving 798 lesions (252 MPNSTs and 546 BPNSTs) were included in the analysis. Pooled and weighted sensitivity, specificity, and AUC values for MRI in detecting MPNSTs were 68% (95% CI, 52-80%), 93% (95% CI, 85-97%), and 0.89 (95% CI, 0.86-0.92) when using feature combination and 88% (95% CI, 74-95%), 94% (95% CI, 89-96%), and 0.97 (95% CI, 0.95-0.98) using diffusion restriction with or without feature combination. Subgroup analysis, such as patients with neurofibromatosis type 1 (NF1) versus those without NF1, could not be performed because of insufficient data. Risk of bias was predominantly high or unclear for patient selection, mixed for index test, low for reference standard, and unclear for flow and timing. CONCLUSION. Combining features such as diffusion restriction optimizes the diagnostic accuracy of MRI for detecting MPNSTs. However, limitations in the literature, including variability and risk of bias, necessitate additional methodologically rigorous studies to allow subgroup analysis and further evaluate the combination of clinical and MRI features for MPNST diagnosis.

Janus Kinase Inhibitors and Risk of Venous Thromboembolism: A Systematic Review and Meta-analysis.

OBJECTIVE: To assess the risk of venous thromboembolism (VTE) in patients treated with Janus kinase (JAK) inhibitors in clinical trials. PATIENTS AND METHODS: We performed a literature search of Ovid MEDLINE and ePub Ahead of Print, In-Process & Other Non-Indexed Citations, and Daily; Ovid EMBASE; Ovid Cochrane Central Register of Controlled Trials; Ovid Cochrane Database of Systematic Reviews; and Scopus, from inception to December 4, 2019, for randomized, placebo-controlled trials with JAK inhibitors as an intervention and reported adverse events. Odds ratio with 95% CI was calculated to estimate the VTE risk using a random effects model. Two independent reviewers screened and extracted data. The GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach was used to assess certainty in estimated VTE risk. RESULTS: We included 29 trials (13,910 patients). No statistically significant association was found between use of JAK inhibitors and risk of VTE (odds ratio, 0.91; 95% CI, 0.57 to 1.47; P=.70; I2=0; low certainty because of serious imprecision). Results using Bayesian analysis were consistent with those of the primary analysis. Results of stratified and meta-regression analyses suggested no interaction by dose of drug, indication for treatment, or length of follow-up. CONCLUSION: We found insufficient evidence to support an increased risk of JAK inhibitor-associated VTE based on currently available data.